This document supplements the material in the "IRIS Media Libraries Programming Guide" (hereinafter, "the Guide"). It adds information about the VL that I think is missing or wrong in the Guide. It doesn't attempt to answer all remaining questions about the VL, or to be a complete reference. This is merely the information that I figured out while making InPerson work on VINO, EV1, Galileo 1.5, and Sirius video for IRIX 5.3 and 6.2. This information applies only to those VL video devices, and not to any other present or future VL video devices.
This document does not represent SGI, and is not an official SGI publication. The entire contents of this document are my personal observations, and/or opinions, and all errors and opinions in it are mine alone. SGI makes no guarantees about anything in this document. Neither do I. You use it at your own risk. Additional disclaimers below.
Some topics NOT discussed in this page include:
The VL_SIZE control on the video source node is a read-only control. The x and y values returned by this control are affected by the setting of the VL_TIMING control on the video source node. The x and y values of this control are not, in general, affected by the settings of any controls in the memory drain node, including VL_ZOOM, VL_SIZE, and VL_CAP_TYPE.
The x component value of this control reveals the width, in pixels, of
the unzoomed, unclipped video input images (fields or frames).
The meaning of the y component value of the video source node's VL_SIZE
control depends on the video device.
On Sirius, the y value is the number of pixel rows in each FIELD,
and includes the count of rows of pixel samples taken from the field's
Vertical Retrace Interval.
On EV1 and VINO, the y value is the number of pixel rows in each FRAME
(pair of fields), and does not include any pixel rows from the Vertical
Retrace Interval.
On all VL video devices except Sirius, VL_FORMAT is not applicable to memory drain nodes, and VL_PACKING is used to select the color space as well as the pattern by which the components are packed into memory buffers. Packings that imply RGB or RGBA color spaces select those spaces. Packings that imply Y, or YUV or YCrCb color spaces select one of those spaces.
A complete description of each of the available color spaces is beyond the scope of this web page. However some discussion of the YUV and YCrCB spaces is necessary. The YUV and YCrCb color spaces are extremely similar to each other, and primarily differ in the ranges of acceptable values for the three components when represented as digital integers.
There is one set of packings that is used with both YUV and YCrCb color spaces. These packings are typically referred to using the letters Y, U, and V, such as "YVYU_422" or "YUV422" (which are common names for the same packing). These names refer to the packing only. The use of a packing named "YUV" or "YVYU" does not imply that the data packed is YUV data, as opposed to YCrCb data. When YCrCb data is being packed with a YUV packing, the Cr component is packed as U, and the Cb component is packed as V. The VL_PACKING_YVYU_422_8 packing is the only packing that is natively supported in hardware (requiring no software conversion) on all VL video devices.
The "422" designation in the packing name means that the pixels are packed so that each horizontally-adjacent pair of pixels share one common set of chrominance (e.g., "UV", or alternately, "CrCb") data. Each pixel has its own value of luminance or "Y" data. So data is packed in pairs of two pixels, two Y values, and one U and one V (or alternately, one Cr and one Cb) value pair, in each pixel pair. In such a pixel packing, the number of pixels in each row will always be even.
The set of VL packings presently defined does not enable the application to choose between the YUV and CCIR-601 YCrCb color spaces. When an app selects VL_PACKING_YVYU_422_8, it will get YUV or YCrCb, depending on the device and the source node from which the data is coming. The Analog-to-Digital converter chip used in VINO and EV1 produces YUV data. Most external digital sources produce YCrCb data. There is no way to tell, from the VL_PACKING control, which of those two spaces (YUV or YCrCb) will be used.
Each of the different VL video devices has a different set of color spaces and packings implemented in hardware. Any other color spaces and/or packings are implemented via software conversion, if they are implemented at all! The table below shows which color-space and packing combinations are implemented in hardware, or software, or not at all, for each device.
Color Spaces and Packings
| VL_PACKING_ | Pixels/ 32-bit word | Format MSB -- LSB |
|---|---|---|
| _RGB_332_P | BBGGGRRR BBGGGRRR BBGGGRRR BBGGGRRR | |
| _RGBA_8 | AAAAAAAA BBBBBBBB GGGGGGGG RRRRRRRR | |
| _RGB_8 | xxxxxxxx BBBBBBBB GGGGGGGG RRRRRRRR | |
| _YVYU_422_8 | UUUUUUUU YYYYYYYY VVVVVVVV YYYYYYYY | |
| _Y_8_P | YYYYYYYY YYYYYYYY YYYYYYYY YYYYYYYY |
Implementations of Color Spaces and Packings
| VL_PACKING_ | Color Space | VINO | EV1 | Galileo 1.5 | SIRIUS |
|---|---|---|---|---|---|
| _RGB_332_P | RGB | H/W | N/A | N/A | N/A |
| _RGBA_8 | RGBA | H/W | S/W | S/W | H/W |
| _RGB_8 | RGB | H/W | S/W | S/W | H/W |
| _YVYU_422_8 | YUV, YCrCb | H/W | H/W | H/W | H/W |
| _Y_8_P | Y | H/W | ??? | ??? | N/A |
The chipset used in VINO and EV1 to convert analog input to digital pixels produces YUV output, not YCrCb output. That is, the values of Y, U and V are in the range 0..255 (the SMPTE YUV ranges), not the smaller 16..235/240 range specified for CCIR601 YCrCb. IndyCam produces CCIR601-compliant YCrCb. For some devices that can't convert color space in hardware, e.g. EV1, the VL converts from YUV to RGBX/RGBA in software. In IRIX 6.2, the VL routines used for this purpose assume the input is CCIR601 YCrCb, not YUV, regardless of what the hardware actually produces. The moral of this story is that if the harware doesn't support the color space you want, and you want the color space conversion to be really accurate, then ask for the pixels in a color space supported by the hardware, and do the color space conversion in your own software, rather than depending on the VL's software colorspace conversion.
With Sirius Video, color space and packing are independent. Color space is chosen by the settings of the VL_FORMAT on the memory drain node, according to table below, and any packing can be applied to any color space, whether it makes sense or not. Color space conversion occurs when the VL_FORMAT of the video source node and the VL_FORMAT of the memory drain node imply different color spaces.
Sirius Color Spaces
| Color Space | VL_FORMAT_ |
|---|---|
| RGB, RGBA | _RGB, _COMPOSITE, _SVIDEO |
| YUV | _BETACAM, _MII, _SMPTE_YUV |
| CCIR601 YCrCb | _FORMAT_DIGITAL_COMPONENT_* |
With Sirius Video, this control is read-only, and is permanently set to VL_CAPTURE_NONINTERLEAVED. Each captured buffer contains exactly one field, unclipped, unzoomed, with N leading pixel rows of samples from the vertical retrace interval.
EV1 implements VL_CAPTURE_NONINTERLEAVED differently from all other VL video devices. On all VL video devices except EV1, when VL_CAP_TYPE is set to VL_CAPTURE_NONINTERLEAVED, each image buffer that the VL gives to the application contains one field, either "even" or "odd", and VL_RATE (the rate at which these buffers are returned) is in fields per second, not frames per second. But on EV1 video devices, when VL_CAP_TYPE is set to VL_CAPTURE_NONINTERLEAVED, each image buffer contains two non-interleaved fields, and VL_RATE is in frames per second.
VINO has a bug that affects capturing with VL_CAPTURE_INTERLEAVED. The first buffer returned after the call to vlStartCapture may have only one field in it. The other field may contain all zero values. For continuous transfers, this may go unnoticed, but for VL_TRANSFER_MODE_DISCRETE captures (e.g. of a single image), this will often result in an image in which every other line is black, or green, depending on the color space in use. A missing field can be detected by examining the first pixel or two in several successive pixel rows. The workaround for this problem is to examine the image for the presence of one black field, and one non-black field, and recapturing another buffer if one field is black. Another solution worth exploring is to capture two buffers, instead of one, and then throw the first one away. This might suffice.
There is no single VL_CAP_TYPE that is available, and implemented in the same way, on all VL video devices. VL_CAPTURE_NONINTERLEAVED is available on all devices, but has different meanings on different platforms. VL_CAPTURE_INTERLEAVED, VL_CAPTURE_EVEN_FIELDS, and VL_CAPTURE_ODD_FIELDS are available and common to all VL video devices except Sirius.
On all VL video devices except Sirius, this control sets the target rate (upper bound) of image buffers per second that will be captured and returned to the application. That is, the VL will not deliver more buffers per second than the rate you specify, but it might deliver less than you specify, especially on EV1.
The contents of each image buffer will be either a frame or a field, as determined by the VL_CAP_TYPE control. Accordingly, VL_RATE is in units of fields per second or frames per second, as determined by the VL_CAP_TYPE control.
VL_RATE is a fraction. Both the numerator and denominator must be specified. The usual value for the denominator is 1. Some devices, e.g. EV1, convert the fraction to a whole integer (truncating, not rounding) of images per second, so using values that are equivalent to integer values is the safest thing to do.
Because VL_RATE is a fraction, vlGetControlInfo() cannot be used to obtain the minimum or maximum values for VL_RATE.
Acceptable values are determined from the following table for non-Sirius devices:
| VL_CAPTURE_NONINTERLEAVED (all devices except EV1 and Sirius) | NTSC | All multiples of 10 and 12 between 10 and 60 |
| PAL | All multiples of 10 between 10 and 50 | |
| VL_CAPTURE_NONINTERLEAVED (EV1) | NTSC | All multiples of 5 and 6 between 5 and 30 |
| PAL | All multiples of 5 between 5 and 25 | |
| VL_CAPTURE_INTERLEAVED, VL_CAPTURE_EVEN_FIELDS, and VL_CAPTURE_ODD_FIELDS | NTSC | All multiples of 5 and 6 between 5 and 30 |
| PAL | All multiples of 5 between 5 and 25 |
With Sirius Video, this control is read-only. Its value is determined by
the setting of the VL_TIMING control on the memory node.
| VL_CAPTURE_NONINTERLEAVED | NTSC | 60 fields per second |
| PAL | 50 fields per second |
VINO's VL_RATE cannot be set to a value less than 5/1.
VINO has hardware bugs that affect decimation of images at these decimation factors: 1/4, 1/5, 1/6, 1/7, and 1/8. These bugs include:
VINO:
| 1/1, 1/2, 1/3 | Implemented in hardware, looks OK. |
| 1/4, 1/6 | Implemented partially in hardware, partially in software. Looks OK, but slower and uses 10% of an R4600 CPU. |
| 1/5, 1/7, 1/8 | implemented in hardware. Looks bad. Green shift. |
EV1:
| 1/1, 1/2, 1/4, 1/8 | Works OK for vid-to-mem |
| 1/3, 1/5, 1/7 | Works only for vid-to-screen, not vid-to-mem, and only with VL_CAPTURE_INTERLEAVED. |
| 2/1, 4/1 | Works only for vid-to-screen, not vid-to-mem. |
Sirius and Galileo 1.5:
| 1/1 | Sirius and Galileo 1.5 don't zoom. |
The VL_SIZE control on the memory drain node determines the number of rows of pixels, and the number of pixels in each row, in each image buffer (field or frame) that the VL returns to the application. If zooming (decimation) is being done, the VL_SIZE control on the memory drain node specifies the size of the image after it has been decimated.
The VL_SIZE control on the memory drain node can be used to "clip" a region out of an image by setting the X and/or Y components to values that are smaller than the size of the captured (and decimated, if applicable) image.
When the (possibly decimated) image is being clipped, the clipped region does not have to come from the upper left hand corner of the (possibly decimated) source image. The VL_OFFSET control on the memory drain node determines the number of top pixel rows to skip and the number of leading pixels to skip in each row to find the first pixel in the (possibly decimated) image to place in the image buffer, the first pixel of the clipping region.
The Guide says, incorrectly, that "VL_OFFSET operates on the unzoomed image; it does not change if the zoom factor is changed.". The truth is that when zooming (decimation) is being used, VL_OFFSET is always in coordinates of the zoomed image. It is as if the entire source image is decimated down, and then the clipping function is applied to the decimated image. The hardware rarely works that way, and usually clips before decimating, but the VL API always specifies the VL_OFFSET in the coordinates of the decimated (virtual) image.
On all VL devices except Sirius, the vertical (Y) component of VL_OFFSET may be specified with a negative value. This causes the clipping region to include row of samples taken before the top of the image, e.g. rows from the Vertical retrace interval. This feature is usually used with VL_ZOOM of 1/1, since the information in the Vertical Retrace Interval isn't an image and doesn't make sense to decimate or average, at least not in the vertical direction.
The VL imposes these requirements on the values of VL_OFFSET and VL_SIZE:
VL_OFFSET and VL_SIZE cannot be both set in one atomic operation. A change in either component of either control could violate one of the rules above (or below), especially after VL_ZOOM is set to a smaller fraction. It may be necessary to alternately and repeatedly set VL_OFFSET and VL_SIZE until no VLValueOutOfRange errors are reported. Follow this link to see a code sample that does this.
Every VL video device places additional limitations on the range of acceptable values of VL_SIZE and VL_OFFSET. Each device has different limitations.
Sirius Video doesn't support preemption. EV1 Video allows preemption in some "sync modes" but not in others. When preemption is not allowed, the VL behaves as if the path was in mode VL_LOCK instead of VL_SHARE; preemptions simply don't happen.
When preemption occurs, your program will receive a VlStreamPreempted
event. This means the following things:
The VLStreamAvailable events create a race between the preempted programs. The outcome of the race depends on the actions taken by the programs, and also on the device's implementation. Generally, as long as all the programs are attempting shared use of the path, the last caller of vlSetupPaths() wins.
When a program receives the VLStreamAvailable event, to regain the usage of the path's data stream, it should call vlSetupPaths(..., VL_SHARE, VL_SHARE) again. This call to vlSetupPaths() to regain access to the path's data stream can succeed or it can fail. The VLStreamAvailable event does not guarantee that it will succeed. Let's look at these cases separately.
If the call to vlSetupPaths() succeeds, the application should set all the controls for the path to the values they had before it was preempted. Your application, that is now regaining the use of the stream, should not assume that the controls are as they were before the preemption. The preemption caused the usage of the path's controls to become VL_SHARE, so the preempting application could have changed any or all of those controls.
Since preemption does not deregister or destroy the ring buffer used by your path, you may resume usage of that same buffer, provided that all the path's controls are returned to the same values they had before the preemption (especially VL_SIZE and VL_PACKING). However, you might want to be doubly sure that the ring buffer is empty before restarting the transfer with a call to vlBeginTransfer().
If the call to vlSetupPaths() to regain acceess to the path's data stream fails (returns a negative value), the path should remain in the same mode in which it was before the failed call to vlSetupPaths(). If your program hasn't changed the path since it was preempted, and if the call to vlSetupPaths() failed with vl error VLPathInUse, the path should still be in the mode in which it was placed by the original preemption, namely as if vlSetupPaths(..., VL_SHARE, VL_READ_ONLY) had been done on this path. The path should continue to behave exactly as when it was preempted, and another VLStreamAvailable should be generated, if and when the path becomes available again.
A VL application that wishes to use the default input source on its device calls vlGetNode(vlServer, VL_SRC, VL_VIDEO, VL_ANY).
Each device's default input source node may be changed via a control. The Video Control Panel supports this, and is usually used to make this choice. When the default input source on any VL video device is changed, (e.g. between a digital input source node and an analog input source node), the VL sends a VLDefaultSource event to all programs that have selected this event, regardless of whether they are using the device whose default input changed, or another VL device. A change in input source between jacks connected to the same input node usually results in a VlControlChanged event, not a VLDefaultSource event. No node changes are forced upon any VL transfers that are in progress. A program that wishes to switch to the new video input source node must take steps to make the switch itself.
When a program receives the VLDefaultSource and wishes to switch to the new default input source, it must take these steps:
While the path's stream usage is in VL_READ_ONLY mode, a VLStreamAvailable event should be generated when the data stream for the selected path becomes available. That is, a program should be able to put itself into a "voluntarily preempted" mode by putting the path's stream usage is in VL_READ_ONLY mode. Whether this actually works or not depends on the device, and on the OS release. It has worked on EV1 at one time. Sirius does not support putting a path into VL_READ_ONLY mode. Sirius also does not support preemption.
An application should be able to transition the stream usage of a path from any one to any other of the four modes (VL_SHARE, VL_READ_ONLY, VL_LOCK, VL_DONE_USING) as many times as it wishes. VL_DONE_USING is not the "mode of no return". The principal difference between VL_READ_ONLY and VL_DONE_USING seems to be the receipt of VLStreamAvailable events in the VL_READ_ONLY mode.
Since Sirius doesn't support VL_READ_ONLY, an application cannot use VL_READ_ONLY to put itself into a mode where it will be notified when another device has finished with the path. A Sirius app has no choice but to go to VL_DONE_USING mode when relenquishing the path, and to use some external or manual means to know when to attempt to reaquire the path.